Virtual frame buffer system and method

ABSTRACT

A system for encoding and decoding a frame (also file), such as a video, graphic, media, or other frame or data, representing a real-time graphic output from a frame buffer, output by a video camera, or another file or data. The file includes frames each comprising macroblocks. Reference frame buffers (PFTs), virtual frame buffer tables (VFTBs) of equal number to the PFTs, each VFTB corresponds to a respective PFT, and respective sectors of each PFT for respective macroblocks are created. Frames of the file are encoded/decoded by successive encode/decode of macroblocks. A pointer is created in the VFBT associated with the PFT rather than encoding/decoding any matching macroblock. The pointer and its reference are relied on for each already encoded/decoded macroblock retained in the PFT. Processing, memory, bandwidth and power requirements for encoding or decoding are reduced.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation and has benefit of priority ofU.S. patent application Ser. No. 15/492,212, titled “Virtual FrameBuffer System and Method”, filed Apr. 20, 2017, which priorityapplication is a continuation of and has benefit of priority of U.S.patent application Ser. No. 14/331,358, titled “Virtual Frame BufferSystem and Method”, filed Jul. 15, 2014 and issued on Sep. 12, 2017 asU.S. Pat. No. 9,762,922, which priority application is a continuation ofand has benefit of priority of U.S. patent application Ser. No.13/176,843, titled “Virtual Frame Buffer System and Method, filed Jul.6, 2011 and issued on Sep. 2, 2014 as U.S. Pat. No. 8,824,560 (which hasbenefit of priority of U.S. Provisional Patent Application No.61/361,969, titled “Virtual Frame Buffer Systems and Methods”, filedJul. 7, 2010). The priority application U.S. patent application Ser. No.15/492,212 is co-pending and has at least one same inventor of thepresent application and is herein incorporated by this reference.

TECHNICAL FIELD

The present invention generally relates to video encode and decodedevices and processes, and more particularly relates to video encode anddecode devices and processing of computing graphics which are relativelystatic in nature, and reducing processing, bandwidth, memory and powerrequirements in such devices and processing.

BACKGROUND

In the process for communicating digitized video over a network from onepoint/node to another point/node, the video is encoded at the senderside device, communicated over the network from the sender side deviceto the recipient side device, and then decoded at the recipient sidedevice. This encoding/decoding reduces the quantity of data that must becommunicated to deliver the video from the sender to the recipient, andtherefore reduces the amount of bandwidth of the network required forthe communication. Typically, video is encoded/decoded according to aspecified technical standard, such as MPEG1, MPEG2, H.263, VC-1 andH.264.

These encode/decode standards are somewhat similar in process and devicerequirements. In particular per each standard, digitized video is storedas a series of “frames” in multiple “frame buffers” in memory, and eachframe buffer for each frame is segregated into a plurality of smaller“macroblock” segments. Encoding and decoding operations each includeprocessing the frames in sequence, via respective sequential processingof the frames via frame buffers for each frame on a macroblock bymacroblock basis for each frame. For video comprising at least somefairly same or static data from time-to-time among respectivemacroblocks (i.e., either of same or other frames), much of theprocessing required for encoding and decoding is primarily mere copyingof data contents. In particular, data of each static macroblock of aframe buffer memory location must be copied into another frame buffermemory location, and this process repeated for every static macroblockof each frame buffer. Both the encoder of the encode side device and thedecoder of the decode side device must typically perform these copy andsimilar operations for each static macroblock, and for all staticmacroblocks of each frame buffer and frame of the series for the entirevideo. Thus, these copy operations consume processing time and capacity.

It would therefore be desirable, and a significant improvement in theart and technology, to provide more efficient encode and decode devicesand processes, particularly for fairly same or static data (e.g.,representing video, graphics, and similar and other information sets andfiles) and other instances.

SUMMARY

Embodiments of the invention include devices and methods for encoding ordecoding a file, such as a video, graphic, media, or other filecommunicated over a network or otherwise, with reduced processing,memory, bandwidth and power requirements. In the embodiments, thedevices and methods either encode/decode each next successive macroblockof a frame of the file or, if that macroblock matches any macroblockthat has already been encoded/decoded (as applicable) of a prior frameand which is then-retained in a reference frame buffer, create and storea pointer in a virtual frame table buffer that points to the previouslyencoded/macroblock already in the reference frame buffer, rather thanencode/decode the macroblock for encoding and/or decoding operations.

An embodiment of the invention is a device for encoding or decoding afile. The file includes frames and each frame comprises macroblocks. Thedevice includes memory, a processor communicatively connected to thememory, an encoder communicatively connected to and controlled by theprocessor, a plurality of reference frame buffers created in the memoryby the encoder, where each reference frame buffer is distinct from anyother reference frame buffers in the memory, virtual frame buffertables, equal in number to number of the reference frame buffers in thememory, created in the memory by the encoder/decoder, each virtual framebuffer table, respectively, corresponds to one reference frame buffer,sectors of each reference frame buffer, each sector is capable ofstoring one of the macroblocks of one frame of the file, and arespective pointer created by the encoder/decoder if any macroblock forencoding/decoding (of any frame) matches any macroblock of another framepreviously encoded/decoded and the encoded/decoded macroblock isthen-retained in one of the reference frame buffers. The encoder eitherencodes each next macroblock of a frame or, if that next macroblockmatches any macroblock that was previously encoded and if the encodedmacroblock is then-retained in any of reference frame buffers, theencoder instead creates the respective pointer and stores the respectivepointer in the virtual frame buffer table corresponding to the frame ofthat next macroblock.

Another embodiment of the invention is a system according to theforegoing encoder or decoder, wherein at least certain of the encoder isthe decoder, and vice versa, and the system further includes acommunication network, a first device connected to the communicationnetwork, the first node includes at least portions of the encoder, and asecond device connected to the communication network, the second nodeincludes at least portions of the decoder. The encoded file iscommunicated by the first node to the second node over the network fordecoding.

Another embodiment of the invention is a system including the encoderand decoder just described above, and also including a communicationlink between the encoder and the decoder.

Yet another embodiment of the invention is a system including theencoder and decoder described above, where at least certain of theencoder is the decoder, and vice versa.

Another embodiment of the invention is a method of encoding a file. Thefile comprises frames and each frame comprises macroblocks for encoding.The method includes creating in a memory a plurality of reference framebuffers, wherein each reference frame buffer is distinct from any otherof the plurality of reference frame buffers in the memory, eachreference frame buffer capable of containing one of the frame, creatingin the memory a plurality of virtual frame buffer tables, equal innumber to number of the reference frame buffers in the memory, eachvirtual frame buffer table, respectively, corresponds distinctly to oneof the reference frame buffer, respectively, creating a plurality ofrespective sectors of each reference frame buffer, respectively, eachrespective sector sufficient for storage of one of the macroblockscorresponding to part of one of the frame of the file, encoding one ofthe frames of the file, by successively encoding each respective one ofthe macroblocks of the one of the frame, unless any already encodedmacroblock (if any) then-retained in any of the respective sectors ofany of the reference frame buffers would be the same as the encodedmacroblock, once encoded in the step of encoding, storing respectiveones of the encoded macroblocks, each respectively corresponding torespective ones of the macroblocks of the one of the frame of the filefrom the step of encoding, in respective ones of the plurality ofrespective sectors, except not storing one of the encoded macroblocks ifany already encoded macroblock (if any) then-retained in any of therespective sectors of any of the reference frame buffers would be thesame as the encoded macroblock, once encoded in the step of encoding,creating a respective pointer in the virtual frame buffer tableassociated with the reference frame buffer then available for the stepof encoding, for each macroblock (if any) of the frame then-currentlyfor the step of encoding, if any macroblock of one of the framepreviously encoded matches the macroblock of the step of creating arespective pointer and if the encoded macroblock of the frame previouslyencoded is then-retained in any of the reference buffer tables, storingthe respective pointer from the step of creating the respective pointer(if performed), in the virtual frame buffer table associated with thereference frame buffer for the step of encoding, and returning to thestep of encoding, but rather than encoding the macroblock in the step ofencoding, relying on the respective pointer for encoding in order toreference the encoded macroblock. The encoder during encoding eitherencodes each next one of the macroblock of the frame or, if anymacroblock of another one of the frames previously encoded isthen-retained in any of the plurality of reference frame buffers,creates the respective pointer and stores the respective pointer in thevirtual frame buffer table corresponding to the frame then beingencoded.

Another embodiment of the invention is a method of decoding an encodedfile. The encoded file comprises encoded frames and each encoded framecomprises encoded macroblocks for decoding and the file, when decoded,comprises frames and each frame comprises macroblocks. The methodincludes creating in a memory a plurality of reference frame buffers,wherein each reference frame buffer is distinct from any other of theplurality of reference frame buffers in the memory, each reference framebuffer capable of containing one of the encoded frame, creating in thememory a plurality of virtual frame buffer tables, equal in number tonumber of the reference frame buffers in the memory, each virtual framebuffer table, respectively, corresponds distinctly to one of thereference frame buffer, respectively, creating a plurality of respectivesectors of each reference frame buffer, respectively, each respectivesector sufficient for storage of one of the macroblocks corresponding topart of one of the frame of the file, decoding one of the encoded framesof the encoded file, by successively decoding each respective one of theencoded macroblocks of the one of the encoded frame, unless the encodedmacroblock, once decoded, would match any macroblock previously decodedand then-retained in any of the respective sectors of any of thereference frame buffers, storing respective ones of the macroblocks,each respectively corresponding to respective ones of the macroblocks ofthe one of the frame of the file from the step of decoding, inrespective ones of the plurality of respective sectors, except notstoring one of the macroblocks if the macroblock, once decoded, wouldmatch any macroblock previously decoded and then-retained in any of therespective sectors of any of the reference frame buffers, creating arespective pointer in the virtual frame buffer table associated with thereference frame buffer then available for the step of decoding, for eachencoded macroblock (if any) of the encoded frame then-currently for thestep of decoding if the encoded macroblock, once decoded, would matchany macroblock previously decoded and retained in any of the referencebuffer tables, storing the respective pointer from the step of creatingthe respective pointer (if performed), in the virtual frame buffer tableassociated with the reference frame buffer for the step of decoding, andreturning to the step of decoding, but rather than decoding the encodedmacroblock in the step of decoding, relying on the respective pointerfor decoding in order to reference the macroblock. The decoder duringdecoding either decodes each next encoded macroblock of the encodedframe or, if any encoded macroblock of another encoded frame previouslydecoded is then-retained in any of the plurality of reference framebuffers, instead creates the respective pointer and stores therespective pointer in the virtual frame buffer table corresponding tothe encoded frame then being decoded.

Another embodiment of the invention is an encode device operating inaccordance with the foregoing method of encoding a file.

Yet another embodiment of the invention is a decode device operating inaccordance with the foregoing method of decoding an encoded file.

Another embodiment of the invention is an encoder/decoder device forprocessing a single reference frame. The encoder/decoder device includesa virtual frame buffer table of a single bit array in a memory and aprocessor connected to the virtual frame buffer table for writing thesingle reference frame to the single bit array.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and notlimitation in the accompanying figures, in which like referencesindicate similar elements, and in which:

FIG. 1 illustrates an exemplary system of three frame reference buffersfor video encoding and decoding, according to certain embodiments of theinvention;

FIG. 2 illustrates exemplary functional elements of the system of FIG.1, according to certain embodiments of the invention;

FIG. 3 illustrates an exemplary method for video encoding and decoding,according to certain embodiments of the invention;

FIG. 4 illustrates another exemplary system of two reference framebuffers for video encoding and decoding via low-end elements, accordingto certain embodiments of the invention; and

FIG. 5 illustrates a system including an encoder communicativelyconnected by a communication network to a decoder, according to certainembodiments of the invention.

DETAILED DESCRIPTION

Referring to FIG. 1, a typical video encoding/decoding system 10illustrates pixels representing a video file (Video In) 102 input to avideo encoder 103 on a frame-by-frame basis for the video file. Forpurposes of this disclosure, the terms “video file”, “video graphic”,“graphic” or “file” means a set, series, collection or combination (orsubset or portion thereof) of video frames stored or storable in anon-transitory tangible media, the video frames representing all or aportion of a graphic, media, video, picture, or other display of aprocessor device, such as a computer, smart phone, tablet computer,laptop computer, or other computing or display rendering device. Videofiles and video frames thereof, for example, include (among others)real-time frame buffer contents that are output of a graphics processorunit (either hardware, software, or combinations), real-time framebuffer contents that are output of a video camera or video device, orother buffer or memory contents read or readable from a storage device,such as hard drive, flash memory, or other tangible media.

An example of each video frame of the video is a respective 48 by 48pixel graphic. When encoding video, the video encoder 103 processinglyencodes the video converting it from the input format to another format,for example, for purpose of standardization, speed, security and/orcompression as will be understood, to create an encoded video file 104.The video encoder 103 also creates and stores reference frame(s) of theencoded video file 104 in one or more reference frame buffers 100 inmemory, for example, Reference Frame Buffers 100 a, 100 b, 100 c, inprocess of encoding. Other numbers of reference frame buffers 100 arepossible for encoding operations, and the reference frame buffers 100are intended only as examples for purposes of discussion. To encode thevideo, the video encoder 103 processes each frame of the video file 102,typically, in succession, progressing through each frame in respectiveblocks of data representing pluralities of distinct sub-portions of eachframe of the video. These blocks of data are referred to, for purposesherein, as macroblocks, and each as a macroblock. Therefore, as thevideo encoder 103 processes and stores the processed frames of the videoin the reference frame buffers 100, the processed frames are stored,macroblock by macroblock of each frame, by the encoder 103 in storagememory (not shown in detail in FIG. 1).

An example macroblock size is 16×16 pixels and, for this example, a48×48 pixel graphic would represent 3 (48/16)×3 (48/16) or 9 totalmacroblocks (e.g., Reference Frame Buffer 100 a is, for purposes of theexample, segregated into nine macroblocks, of which macroblock 101 isone). Each macroblock occupies a specific block of memory for theapplicable Reference Frame Buffer 100 a, 100 b or 100 c. In the case ofmacroblock 101 and example of 16×16 pixels, the macroblock 101 (whenrepresented in a typical video format) requires 384 bytes of memory ineach of the Reference Frame Buffers 100 a, 100 b, 100 c. The totalmemory for each Reference Frame Buffer 100 a, 100 b, 100 c, therefore,is 384 bytes×9 for a total size of 3456 bytes of memory.

The video encoder 103 processes the video creating Reference FrameBuffers 100 a, 100 b, 100 b and using these to encode the video,creating as output the encoded video file 104. The encoded video 104output by the encoder 103 is stored (not shown in detail in FIG. 1)and/or delivered (e.g., communicated) to a video decoder 106, such asvia a network, circuit(s), or otherwise. The video decoder 106 receivesinput of the encoded video 104 and processes it by converting theencoded video 104 back to the decoded format of the video file 102originally input to the encoder 103 (or another decoded format for useby the recipient side, as will be understood), creating a video file(Video Out) 107 of the decoded format as output. The decoding processincludes, for example, decompressing, reversing or changing technicalstandard, permitting, and/or other operability or access presenting atthe recipient side as will be understood. In the process of decoding,the video decoder 106 creates Reference Frame Buffers 105 a, 105 b, 105c in memory of same number and size as those of the Reference FrameBuffers 100 a, 100 b, 100 c of the encoder 103. The video is decoded bythe video decoder 106 on a frame by frame, macroblock per macroblock perframe, basis, and the video decoder 106 outputs each video frame, i.e.,as the Video Out 107. At respective given points during decoding,contents of the Reference Frame Buffers 105 a, 105 b and 105 c for thedecoder 106 are the same as the contents of the Reference Frame Buffers100 a, 100 b, 100 c, respectively, of the encoder 103 during encoding atcorresponding points (but for encode purposes, rather than decodepurposes).

Referring now to FIG. 5, according to certain embodiments, a system 50includes an encode device 502 and a decode device 504. The encode device502 and the decode device 504 are communicatively connected by acommunication link, for example, a wired, wireless, cellular, optical,satellite or other communication network 506, combinations of these, orconnected/interconnected networks such as any combinations of these. Theencode device 502 includes a video encoder 510 communicatively connectedto memory 503. The video encoder 510 is, includes, or is controlled by aprocessing device, for example, the video encoder 510 is amicroprocessor, software program stored in memory, logic circuit, orcombinations of these, or other processor(s). The memory 503 is anywritable data storage, for example, random access memory (RAM),harddrive, floppy drive, compact disk (CD), flash drive, digital videodisk (DVD), or combinations of these, or any other writable datacontainer(s). The decode device 504 includes a video decoder 516communicatively connected to memory 517. The video decoder 516 is,includes or is controlled by a processing device. For example, the videodecoder 510 is a microprocessor, software program stored in memory,logic circuit, or combinations of these, or other processor(s). Thememory 517 is any writable data storage, for example, random accessmemory (RAM), harddrive, floppy drive, compact disk (CD), flash drive,digital video disk (DVD), or combinations of these, or any otherwritable data container(s).

The video encoder 510, in operation with the memory 503, processinglyencodes a digitized video file representing a playable video. A sourceof the digitized video file is, for example, a frame buffer of agraphics card or video camera or of a memory or storage media, thecontents of which are output to the video encoder 510. When encoding thevideo, the video encoder 510 processingly creates in the memory 503 aplurality of reference frame buffers (hereafter referred to forconvenience as “physical frame buffers” or (PFBs) to distinguish fromthe Reference Frame Buffers of FIG. 1) and a plurality of respectivevirtual frame buffer tables (VFBTs). For purposes of example, FIG. 5illustrates three PFBs 512 a, 512 b, and 512 c and three VFBTs 514 a,514 b, 514 c. The number of the virtual frame buffer tables/VFBTs is thesame number of the reference frame buffers/physical frame buffers/PFBs.Although three PFBs and three VFBTs are illustrated and described forexample purposes, the number of PFBs and VFBTs will in each instance bethe same (except as per certain alternative embodiments later describedwith respect to FIG. 4), with each PFB having a corresponding VFBT forsuch PFB.

The video encoder 510 then processingly encodes a first frame of thevideo file, by encoding, on a macroblock by macroblock basis, respectivemacroblocks of the first frame, creating respective encoded macroblocks(not shown in detail in FIG. 5) for the first frame via one of theVFBTs, for example, VFBT 512 a, and storing the encoded macroblocks inone of the PFBs for the first frame, for example, in PFB 514 a. Eachpointer element of the VFBT 512 a now identifies PFB 514 a. The videoencoder 510 then next proceeds to processingly encode a second frame ofthe video file, for example, via VFBT 512 b and PFB 514 b, macroblock bymacroblock. For macroblocks that are not then-contained in any PFB 514a, 514 b, 514 c, the macroblocks are saved in the PFB 514 b by the videoencoder 510 and respective pointer elements therefor are saved in theVFBT 512 b to identify PFB 514 b as containing the saved macroblocks,respectively. For any macroblock of the second frame that is duplicateof a previously encoded macroblock (the video encoder 510 detects anyduplicate; which would not be the case in encoding of the first frame,but can be the case with the second or subsequent frames of thedigitized video file) then-retained in any PFB (i.e., in the case of thesecond frame, any match already in the PFBs could only be in PFB 514 a),rather than encoding and saving that duplicate macroblock in the PFB 514b, the video encoder 510 instead sets a pointer element (not shown indetail in FIG. 5) stored in the VFBT for the second (or othersubsequent) frame (i.e., in the VFBT 512 b in the example) identifyingthe macroblock of the PFB 514 a which would otherwise be duplicated(i.e., rather than encoding and storing in PFB 514 b). The pointerelement represents an identifier of a PFB that contains the encodedmacroblock corresponding to the duplicate macroblock that was previouslyencoded, and also of a particular PFB sector (not shown in detail inFIG. 5) containing the previously encoded macroblock of that PFB.

The video encoder 510 continues processingly encoding each next frame ofthe digitized file in succession or, if any macroblock(s) of the frameis duplicate of any previously encoded macroblock then retained in anyof the PFBs 512 a, 512 b, 512 c, skipping encoding and storing of themacroblock and instead creating the respective pointer element for thepreviously encoded macroblock (and PFB sector and PFB thereof) andstoring it in the VFBT for the particular frame. As encoding of eachrespective video frame is performed by the video encoder 510, the videoencoder 510 can store and/or communicate the encoded video frame fromthe video encoder 510, or otherwise deliver or make available the framefrom the encode device 502, for decoding by the decode device 504.Alternately, if desired in the embodiment, the video encoder 510completes encoding of the video file in entirety (or in portion, ifapplicable in the embodiment, for example, where portions arereconstituted as the whole or otherwise at a decoder), and thencommunicates or stores the video, as applicable.

According to certain embodiments, the encoded video file from the videoencoder 510 is communicated from the encode device 502 to the decodedevice 504, via the communications network 506. The decode device 504,in operation with the memory 517, processingly decodes the encoded videofile created by the video encoder 510. In decoding, the video decoder516 of the decode device 504 processingly creates in the memory 517 aplurality of reference frame buffers, i.e., PFBs, distinct from those ofthe encode device 502. The video decoder 516 also processingly createsin the memory 517 a plurality of respective virtual frame buffer tables,i.e., VFBTs, distinct from those of the encode device 502. As anexample, FIG. 5 illustrates three PFBs 518 a, 518 b, and 518 c and threeVFBTs 520 a, 520 b, 520 c of the decode device 504. As with the encodedevice 502, the decode device 504 has same number of the VFBTs as numberof the PFBs Although three PFBs and three VFBTs of the decode device 504are illustrated and described for example purposes, the number of PFBsand VFBTs will in each instance be the same and the number of PFBs (andconsequently of VFBTs) equals the number of the PFBs (and VFBTs) of theencode device 502.

Continuing in operation, the video decoder 516 processingly decodes afirst frame of the encoded video file, by decoding, on a macroblock bymacroblock basis, respective macroblocks of the first frame via one ofthe PFBs and its VFBT, creating respective decoded macroblocks (notshown in detail in FIG. 5) for the first frame in, for example, via theVFBT 520 a and the PFB 518 a. The video decoder 516 proceeds,processingly decoding and storing a second frame of the digitized videofile via another VFBT and PFB, for example, VFBT 520 b and PFB 518 b,for the second frame, macroblock by macroblock; however, rather thandecoding and storing any macroblock of the second encoded frame that isduplicate of a previously decoded macroblock then-retained in any otherPFB (which would not be the case in encoding of the first frame, but canbe the case with the second or subsequent frames of the encoded videofile), that duplicate macroblock is not decoded, and instead, the videodecoder 516 creates a pointer element (not shown in detail in FIG. 5;i.e., which pointer element of the decode device 504 is distinct fromthe pointer element of the encode device 502) and stores the pointerelement in the VFBT 520 b for the second (or other subsequent) frame.The pointer element of the decode device 504 represents an identifier ofthe PFB 518 a of the decode device 504 that contains the duplicatemacroblock that was previously decoded, and also of a particular PFBsector (not shown in detail in FIG. 5) of that PFB 518 a which containsthe previously decoded macroblock for which the presently processedmacroblock is duplicate.

The video decoder 516 continues processingly encoding each next frame ofthe encoded video file in succession (e.g., overwriting respective PFBsfor frames after the third frame and storing or dispensing decodedframes of PFBs and VFBTs in storage memory or otherwise prior tooverwriting) or, if any encoded macroblock(s) in the particular frameare same as then-retained in any of the PFBs from a prior frame whichhas already been decoded, skipping decoding of the macroblock for theparticular frame and, instead, creating the respective pointer elementfor the previously decoded macroblock (and PFB sector and PFB thereof)and storing the pointer element in the VFBT for the particular frame. Asthe video decoder 516 completes decoding of each frame of the encodedvideo file, the video decoder 516 communicates the decoded frame asoutput of the video decoder 516. For example, each frame, once decoded,is communicated by the video decoder 516 as a stream to a media player,graphics card and/or other computer or processor device (hardware,software or combination) for display or other output on a monitor,screen, display or other output device, or is otherwise processed orstored in tangible media, such as harddrive, memory or the like (notshown in detail in FIG. 5), available for selective used or operationfor display of the video, or otherwise.

Referring to FIG. 2, in conjunction with FIG. 5, according to certainembodiments a system 20 includes the PFBs and the VFBTs asabove-described and now further detailed in encode and decodeoperations. The system 20 also includes a video encoder/decoder 206,which can be either an encoder or a decoder (for example, the videoencoder 510 of the encode device 502 and/or the video decoder 516 of thedecode device 504). Because the embodiments of FIG. 5 and of FIG. 2illustrate three PFBs and three VFBTs, respectively, for encoding anddecoding operations, FIG. 2 for purposes of explanation refers to thevideo encoder/decoder 206 generically, intended as including any of anencoder, decoder or combination, in which the encode and/or decodeprocessing employs PFBs and VFBTs according to the embodiments. Thevideo encoder/decoder 206 are each implemented by creates and uses threevirtual frame buffer tables (VFBTs) 203, 204, 205, and three physicalframe buffers (PFBs) 200, 201, 202. The video encoder/decoder 206communicatively accesses the three physical frame buffers (PFBs) 200,201, 202 and also communicatively accesses the three virtual framebuffer tables (VFBTs) 203, 204, 205.

Each PFB 200, 201, 202 contains nine macroblocks, for example purposesas previously discussed. Therefore each VFBT 203, 204, 205 can includenine entry elements in memory. Each entry element of each VFBT 203, 204,205 points to one of the PFBs 200, 201, 202. For example, entry element207 of VFBT 205 contains a 1 which represents PFB 201. Entry element 207is in the seventh location in the VFBT 205 and therefore corresponds tothe macroblock 208 of the seventh location of the PFB 201. The videoencoder/decoder 206, by reference to a particular one of the entryelements of one of the VFBTs 203, 204 or 205, is thus directed to readcontents of a particular macroblock of particular one of the PFB 200,201, 202.

In operation, all entry elements of each of the VFBT's 203, 204, 205 areinitially set to a special value of “none”. When a first frame (i.e., ofa video graphic) is received for processing, one of the PFB 200, 201, or202 and one of the VFBTs 203, 204, 205 is required by theencoder/decoder 206. The VFBT 203 and the PFB 200 are used: respectivesets of bytes of the first frame are processed and a written torespective macroblocks of the PFB 200, and each entry element of theVFBT 203 is set to 0. This indicates for the encoder/decoder 206 thatthe all of the macroblocks for the VFBT 203 are contained in the PFB200.

When a second frame buffer (i.e., a video graphic comprising a next(second) frame) is received, the encoder/decoder employs the VFBT 204and the PFB 201. As macroblocks are written to the PFB 201, a 1 iswritten to the corresponding entry element of the VFBT 204, representingthat the macroblock is stored in the PFB 201. When the videoencoder/decoder 206 receives an indication that a macroblock matches thealready stored macroblock in PFB 200 a 0 is written to the VFBT 204 bythe encoder/decoder 206. The 0 entry of the VFBT 204 therefore indicatesthat the particular macroblock is contained in the PFB 200, and nomacroblock was written in PFB 201. Thus, if there is match for amacroblock of the second frame, no entry other than the 0 in VFBT 204 isnecessary. Less processing is therefore required.

For a next (third) frame, the PFB 202 and the VFBT 205 similarly operateand VFBT 205 is written with either: a 2 where the macroblock is new(i.e., not then-retained in a PFB as previously encoded/decoded) andwill be written into the PFB 202, a 1 where the macroblock is already inthe PFB 201, or a 0 where the macroblock is already contained in the PFB200. PFB 202 is written with the macroblock only if contents are new(i.e., not already written in PFB because of another macroblockencode/decode operation) and therefore not contained in either PFB 200or PFB 201.

After three frame buffer requests (i.e., three frames received), theprocess continues. The encoder/decoder 206 uses VFBT 0 203 again, nowfor the next frame. Referring to FIG. 2 as an example, for a firstmacroblock, entry element 0 in the VFBT 203 contains a 0, therefore themacroblock is in PFB 200. Entry element 0 in the VFBT 204 contains a 2and entry element 0 in the VFBT 205 contains a 1, therefore, the PFB 200is not referred to by either VFBT 204 or VFBT 205 and therefore is freeto be used by the encoder/decoder 206 to store the first macroblock ofthe relevant frame. A 0 is written to the VFBT 203, location 0,representing that the macroblock is in PFB 200 and the marcoblock iswritten in PFB 200

Next in the example, for a second macroblock, entry element 1 in theVFBT 203 contains a 1; the VFBT 204, entry element 1, also contains a 1;and the VFBT 205, entry element 1, contains a 0. Because the PFB 202 isnot referred to by any other VFBT, the encoder/decoder 206 uses the PFB202 for storage of the second macroblock. The macroblock is written bythe encoder/decoder 206 to the PFB 202, and a 2 is written into the VFBT203, entry element 1, corresponding to the PFB 202.

Continuing for a third macroblock of the relevant frame, entry element 2in the VFBT 203 is 2 (i.e., for PFB 202), VFBT 204 is 1 (i.e., for PFB201), and VFBT 205 is 0 (i.e., for PFB 200). Since the VFBT 203 refersto the PFB 202 and the VFBT 204 and VFBT 205 do not, the PFB 202 is usedby the encoder/decoder 206. The bytes of the macroblock are written bythe encoder/decoder 206 to the PFB 202 and a 2 is written into the VFBT203, entry element 2.

This process is repeated for the entire frame and the entire video, on aframe by frame basis. A destination device of the digitized video filefrom the video decoder 510 is, for example, a display for visuallydepicting the video in real-time or a memory or storage media.

Therefore in processing non-changing macroblocks (i.e., the macroblockhas already been stored in a PFB), only a VFBT entry must be updated.For processing macroblocks that have changed, therefore, at most threeVFBT entries must be read to determine which PBF to use. In comparison,without such embodiments, copying of an entire macroblock from one PFBto another PFB would be necessary. Thus the embodiments limit processingthat would otherwise be required.

Referring to FIG. 3, a method 30 commences with a step 300. In themethod 30, the step 300 is performed once to commence operation onreceipt of a first frame, and steps 301 are performed once for eachframe and steps 302 to 307 are performed once for each macroblock of theframe.

In the step 300, all VFBT entries are set to a not used (NU) value. Foreach successive frame, a VFBT is selected 301 in order of precedence,for example, 0, 1, 2, 0, 1, 2, etc. In a step 302, an encoder or decoderreceives an indication that a marcoblock has changed and has newcontents or has not changed and matches a prior macroblock contained inone of the VBTs. If a match is indicated, an entry element of the PFBthat already contains the new macroblock is written to an applicableVFTB in a step 308. The method 30 then continues to a step 307. If nomatch was indicated in the step 302, then determination is made bysearching each of the VFBTs in a step 303 whether any of the PFBs areunused. If the step 303 determines there is an unused PFB, then a numberidentifier of the unused PFB is written to the VFBT in a step 304. Ifthe step 303 determines that there are no unused PFBs, then a specifiedone of the PFBs referenced by the current VFBT is reused, in a step 305.The particular PFB reused in the step 305 is any PFB which is thenunused by all other VFBTs. In a step 306, following the step 304 or thestep 305, as applicable, the new macroblock is written to the PFBselected in the step 304 or 305, respectively. In a step 307determination is made whether any more macroblock(s) exist in thecurrent frame. If any more macroblock(s) is present, the method 30returns to the step 302; otherwise, if no more macroblock(s) in thisframe, the method 30 returns to the step 301.

Variations to the foregoing embodiments include encoders/decoders thatemploy a single reference frame buffer or different multiples (otherthan three as described above) of reference frame buffers and VFBTs.

Referring to FIG. 4, a system 40 includes a video encoder/decoder 403for communicatively connecting to two reference frame buffers (i.e.,PFBs) 400, 401 and one virtual frame buffer (VFBT) 404 during processingof a video file (not shown). As with the foregoing embodiments, theencoder/decoder 403 creates the PFBs 400, 401 and the VFBT 404 in memory(not shown in detail in FIG. 4) and uses these for processingly encodingor decoding the video file. In particular, the PFB 400 and PFB 401 areemployed, with one containing the reference frame relative to previouslyprocessed frame or frames (as will be understood) of the video file andthe other containing a temporary current frame relative to thethen-presently in process frame of the video file. Only the single VFBT404 is required. Also since the VFBT 404 entry elements represent onlypossible values of either 0 or 1 (i.e., a 0 references PFB 400, a 1references PBF 401), the VFBT 404 is implementable by theencoder/decoder 403 as a bit array, instead of as a set of memorylocations in memory.

Operation is the similar to that described with respect to FIG. 2, butwith two differences (at least). A first difference is that two framebuffers PFB 400 and PFB 401, instead of the three frame buffers PFB 200,PFB 201, PFB 202, are created and used by the encoder/decoder 403 duringencode/decode processing. A second difference is that a second VFBT (notshown in detail in FIG. 4) is merely temporarily created by theencoder/decoder 403 as a copy of the single actual VFBT 404. Thistemporary second VFBT is employed by the encoder/decoder 403 to accessthe reference frame then-retained in the PFB 400 or 401 relative topreviously processed frame or frames. Only the actual VFBT 404 isupdated by the encoder/decoder 403 as frame processing proceeds. At theend of processing of a frame, VFBT 404 contains the VFBT for the singlereference frame just processed. The temporary VFBT can be discarded. Theprocess is repeated with the PFBs 400, 401 and VFBT 403 as to each nextsuccessive frame of the video file, by creation and store by theencoder/decoder 403 of the reference frame in the temporary VFBT andthen discard of the temporary VFBT at end of processing thethen-in-process frame.

In the foregoing specification, the invention has been described withreference to specific embodiments. However, one of ordinary skill in theart appreciates that various modifications and changes can be madewithout departing from the scope of the present invention as set forthin the claims below. Accordingly, the specification and figures are tobe regarded in an illustrative rather than a restrictive sense, and allsuch modifications are intended to be included within the scope of thepresent invention.

Benefits, other advantages, and solutions to problems have beendescribed above with regard to specific embodiments. However, thebenefits, advantages, solutions to problems and device(s), connection(s)and element(s) that may cause any benefit, advantage, or solution tooccur or become more pronounced are not to be construed as a critical,required, or essential feature or element of any or all the claims. Asused herein, the terms “comprises, “comprising,” or any other variationthereof, are intended to cover a non-exclusive inclusion, such that aprocess, method, article, or apparatus that comprises a list of elementsdoes not include only those elements but may include other elements notexpressly listed or inherent to such process, method, article, orapparatus.

What is claimed is:
 1. A device for encoding a file, the file comprisesframes and each frame comprises macroblocks for encoding, comprising:memory; a processor communicatively connected to the memory; an encodercommunicatively connected to and controlled by the processor; aplurality of reference frame buffers created in the memory by theencoder under control of the processor, wherein each reference framebuffer is distinct from any other of the plurality of reference framebuffers in the memory, each reference frame buffer capable of containingone of the frame; a plurality of virtual frame buffer tables, equal innumber to number of the reference frame buffers in the memory, createdin the memory by the encoder under control of the processor; a pluralityof respective sectors of each reference frame buffer, respectively, eachrespective sector capable of storing one of the macroblockscorresponding to part of one of the frame of the file; and a respectivepointer created by the encoder during encoding, if any one of themacroblock of one of the frame of the file matches any macroblock ofanother one of the frames previously encoded by the encoder wherein anyof the plurality of reference frame buffers then-retains the macroblockof the other of the frames; wherein the encoder during encoding eitherencodes each next one of the macroblock of the frame or, if anymacroblock of another one of the frames previously encoded isthen-retained in any of the plurality of reference frame buffers,creates the respective pointer and stores the respective pointer in thevirtual frame buffer table corresponding to the frame then beingencoded.
 2. A device for decoding an encoded file, the encoded filecomprises encoded frames and each frame comprises encoded macroblocksfor decoding, comprising: memory; a processor communicatively connectedto the memory; a decoder communicatively connected to and controlled bythe processor; a plurality of reference frame buffers created in thememory by the decoder under control of the processor, wherein eachreference frame buffer is distinct from any other of the plurality ofreference frame buffers in the memory, each reference frame buffercapable of containing one of the encoded frame; a plurality of virtualframe buffer tables, equal in number to number of the reference framebuffers in the memory, created in the memory by the decoder undercontrol of the processor, each virtual frame buffer table, respectively,corresponds distinctly to one of the reference frame buffer,respectively; a plurality of respective sectors of each reference framebuffer, respectively, each respective sector capable of storing one ofthe encoded macroblocks corresponding to part of one of the encodedframe of the encoded file; and a respective pointer created by thedecoder during decoding, if any one of the encoded macroblock of one ofthe encoded frame of the encoded file matches any encoded macroblock ofanother one of the encoded frames previously decoded by the decoderwherein any of the plurality of reference frame buffers then-retains theencoded macroblock of the other of the encoded frames; wherein thedecoder during decoding either decodes each next one of the encodedmacroblock of the encoded frame or, if any encoded macroblock of anotherone of the encoded frames previously decoded is then-retained in any ofthe plurality of reference frame buffers, creates the respective pointerand stores the respective pointer in the virtual frame buffer tablecorresponding to the encoded frame then being decoded.
 3. A systemaccording to claims 1 and 2, further comprising: a communication linkconnecting the device of claim 1 to the device of claim
 2. 4. The systemof claim 3, wherein the communication link is a circuit.
 5. The systemof claim 3, wherein the communication link is selected from the groupof: wire, wireless link, cellular link, optic link, satellite, publicswitched telephone network, Internet, local area network, wireless localarea network, or other communication network, or any combination of anyor some of these.
 6. A system according to claims 1 and 2, wherein atleast certain of the device of claim 1 is the device of claim 2, andvice versa.
 7. A method of encoding a file, the file comprises framesand each frame comprises macroblocks for encoding, comprising the stepsof: creating in a memory a plurality of reference frame buffers, whereineach reference frame buffer is distinct from any other of the pluralityof reference frame buffers in the memory, each reference frame buffercapable of containing one of the frame; creating in the memory aplurality of virtual frame buffer tables, equal in number to number ofthe reference frame buffers in the memory, each virtual frame buffertable, respectively, corresponds distinctly to one of the referenceframe buffer, respectively; creating a plurality of respective sectorsof each reference frame buffer, respectively, each respective sectorsufficient for storage of one of the macroblocks corresponding to partof one of the frame of the file; encoding one of the frames of the file,by successively encoding each respective one of the macroblocks of theone of the frame, unless any already encoded macroblock (if any)then-retained in any of the respective sectors of any of the referenceframe buffers would be the same as the encoded macroblock, once encodedin the step of encoding; storing respective ones of the encodedmacroblocks, each respectively corresponding to respective ones of themacroblocks of the one of the frame of the file from the step ofencoding, in respective ones of the plurality of respective sectors,except not storing one of the encoded macroblocks if any already encodedmacroblock (if any) then-retained in any of the respective sectors ofany of the reference frame buffers would be the same as the encodedmacroblock, once encoded in the step of encoding; creating a respectivepointer in the virtual frame buffer table associated with the referenceframe buffer then available for the step of encoding, for eachmacroblock (if any) of the frame then-currently for the step ofencoding, if any macroblock of one of the frame previously encodedmatches the macroblock of the step of creating a respective pointer andif the encoded macroblock of the frame previously encoded isthen-retained in any of the reference buffer tables; storing therespective pointer from the step of creating the respective pointer (ifperformed), in the virtual frame buffer table associated with thereference frame buffer for the step of encoding; and returning to thestep of encoding, but rather than encoding the macroblock in the step ofencoding, relying on the respective pointer for encoding in order toreference the encoded macroblock; wherein the encoder during encodingeither encodes each next one of the macroblock of the frame or, if anymacroblock of another one of the frames previously encoded isthen-retained in any of the plurality of reference frame buffers,creates the respective pointer and stores the respective pointer in thevirtual frame buffer table corresponding to the frame then beingencoded.
 8. A method of decoding an encoded file, the encoded filecomprises encoded frames and each encoded frame comprises encodedmacroblocks for decoding and the file, when decoded, comprises framesand each frame comprises macroblocks, comprising the steps of: creatingin a memory a plurality of reference frame buffers, wherein eachreference frame buffer is distinct from any other of the plurality ofreference frame buffers in the memory, each reference frame buffercapable of containing one of the encoded frame; creating in the memory aplurality of virtual frame buffer tables, equal in number to number ofthe reference frame buffers in the memory, each virtual frame buffertable, respectively, corresponds distinctly to one of the referenceframe buffer, respectively; creating a plurality of respective sectorsof each reference frame buffer, respectively, each respective sectorsufficient for storage of one of the macroblocks corresponding to partof one of the frame of the file; decoding one of the encoded frames ofthe encoded file, by successively decoding each respective one of theencoded macroblocks of the one of the encoded frame, unless the encodedmacroblock, once decoded, would match any macroblock previously decodedand then-retained in any of the respective sectors of any of thereference frame buffers; storing respective ones of the macroblocks,each respectively corresponding to respective ones of the macroblocks ofthe one of the frame of the file from the step of decoding, inrespective ones of the plurality of respective sectors, except notstoring one of the macroblocks if the macroblock, once decoded, wouldmatch any macroblock previously decoded and then-retained in any of therespective sectors of any of the reference frame buffers; creating arespective pointer in the virtual frame buffer table associated with thereference frame buffer then available for the step of decoding, for eachencoded macroblock (if any) of the encoded frame then-currently for thestep of decoding if the encoded macroblock, once decoded, would matchany macroblock previously decoded and retained in any of the referencebuffer tables; storing the respective pointer from the step of creatingthe respective pointer (if performed), in the virtual frame buffer tableassociated with the reference frame buffer for the step of decoding; andreturning to the step of decoding, but rather than decoding the encodedmacroblock in the step of decoding, relying on the respective pointerfor decoding in order to reference the macroblock; wherein the decoderduring decoding either decodes each next one of the encoded macroblockof the encoded frame or, if any encoded macroblock of another one of theencoded frames previously decoded is then-retained in any of theplurality of reference frame buffers, creates the respective pointer andstores the respective pointer in the virtual frame buffer tablecorresponding to the encoded frame then being decoded.
 9. The device ofclaim 1, wherein the file is a graphical type screen (such as a wordprocessor or spreadsheet) of size about 5,000 to about 10,000macroblocks; and wherein only about one to about ten macroblocks need beupdated in a changed frame, wherein the processing, memory requirements,bandwidth and power are thereby reduced in encoding or decoding,respectively.
 10. The method of claim 7, wherein the file is a graphicaltype screen (such as a word processor or spreadsheet) of size about5,000 to about 10,000 macroblocks; and wherein only about one to aboutten macroblocks need be encoded in the step of encoding for any frame ofmacroblocks not previously encoded, wherein the processing, memoryrequirements, bandwidth and power are thereby reduced in encoding. 11.The method of claim 8, wherein the encoded file represents a graphicaltype screen (such as a word processor or spreadsheet) of size about5,000 to about 10,000 macroblocks; and wherein only about one to aboutten encoded macroblocks need be decoded in the step of decoding for anyencoded frame of encoded macroblocks not previously decoded of anotherencoded frame, wherein the processing, memory requirements, bandwidthand power are thereby reduced in decoding.
 12. An encode deviceoperating in accordance with claim
 7. 13. A decode device operating inaccordance with claim
 8. 14. An encoder/decoder device for processing asingle reference frame, comprising: a virtual frame buffer table of asingle bit array in a memory; and a processor connected to the virtualframe buffer table for writing the single reference frame to the singlebit array.
 15. The device of claim 1, wherein the device is selectedfrom the group consisting of: circuit, computer program stored inmemory, processor, and any combination of these.
 16. The device of claim2, wherein the device is selected from the group consisting of: circuit,computer program stored in memory, processor, and any combination ofthese.
 17. A system according to claims 1 and 2, wherein at leastcertain of the device of claim 1 is the device of claim 2, and viceversa, and the system further comprises: a communication network; afirst device connected to the communication network, the first nodeincludes at least portions of the device of claim 1; and a second deviceconnected to the communication network, the second node includes atleast portions of the device of claim 2; wherein the encoded file iscommunicated by the first node to the second node over the network fordecoding.
 18. The device of claim 2, wherein the file is a graphicaltype screen (such as a word processor or spreadsheet) of size about5,000 to about 10,000 macroblocks; and wherein only about one to aboutten macroblocks need be updated in a changed frame, wherein theprocessing, memory requirements, bandwidth and power are thereby reducedin encoding or decoding, respectively.